How to install KVM and libvirt on CentOS 6.2 with bridged networking

This is a how to install the KVM hypervisor and libvirt virtualization library on Linux CentOS 6.2. At the end of this guide you will have a CentOS box (name it host) with the following capabilities:

• virtualization capacity: chance for multiple guests (Linux/Windows) running and sharing the host's hardware 
• bridge network configuration: you'll be able to directly access to the guests as if they were physical machines on the same LAN and viceversa.
• visual management: using virt-manager (I use Ubuntu 11.10 on this how to) you will be able to remotely admin the virtual machines on the CentOS host.

What's what?

KVM:

"KVM (for Kernel-based Virtual Machine) is a full virtualization solution for Linux on x86 hardware containing virtualization extensions (Intel VT or AMD-V). It consists of a loadable kernel module, kvm.ko, that provides the core virtualization infrastructure and a processor specific module, kvm-intel.ko or kvm-amd.ko..."
"Using KVM, one can run multiple virtual machines running unmodified Linux or Windows images. Each virtual machine has private virtualized hardware: a network card, disk, graphics adapter, etc."

libvirt:

• "A toolkit to interact with the virtualization capabilities of recent versions of Linux... "
• "A Free software available under the GNU Lesser General Public License."
• "A long term stable C API"
• "A set of bindings for common languages"

Intel-VT and AMD-V

• What is Intel VT / AMD-V / hvm?. 
• How can I tell if I have Intel VT or AMD-V?

Requirements

• Hardware: 
• Processor with support for Intel-VT or AMD-V technology
• Plenty RAM memory depending on the number of guests
• Network connectivity
• Software: 
• A previosly installed Linux CentOS 6.2 with a recent Linux kernel. On this how to:

$ uname -r
2.6.32-220.2.1.el6.x86_64

$ lsb_release -a
LSB Version: :core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 6.2 (Final)
Release: 6.2
Codename: Final

• A Linux client with a Desktop environment installed
• An available CentOS mirror/repository

Installation Steps

1. Login into the CentOS as bozz user (a sudoer user) and check if your hardware support for  virtualization extensions, on my hardware it was:

$ egrep '^flags.*(vmx|svm)' /proc/cpuinfo
flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
flags  : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid

2. Install kvm and libvirt packages:

$ sudo yum install kvm libvirt

3. Update both packages to the latest version available on repositories/mirrors:

$ sudo yum update libvirt kvm
...
$ yum info libvirt
Installed Packages
Name        : libvirt
Arch        : x86_64
Version     : 0.9.4
Release     : 23.el6_2.4
...
$ yum info qemu-kvm
Installed Packages
Name        : qemu-kvm
Arch        : x86_64
Epoch       : 2
Version     : 0.12.1.2
Release     : 2.209.el6_2.1
...

4. Restart the libvirtd daemon:

$ sudo service libvirtd restart

5. Verify if the kvm module is loaded, you should see amd or intel depending on the hardware:

$ lsmod | grep kvm
kvm_intel              50380  0
kvm                   305113  1 kvm_intel

6. Issue a virsh command to ensure local root connectivity first:

$ sudo virsh sysinfo
<sysinfo type="smbios">
...

7. [OPTIONAL] To use KVM by a non-root user verify if kvm group was created on installation:

$ cat /etc/group | grep kvm
kvm:x:36:qemu

then add the bozz user to kvm group, so it can gain access to hypervisor:

$ sudo usermod -a -G kvm bozz
$ logout

login again as the bozz user and verify kvm's membership:

$ id
uid=500(bozz) gid=500(bozz) groups=500(bozz),10(wheel),36(kvm) context=...

and verify if /dev/kvm is owned by group kvm:

$ ls -l /dev/kvm
crw-rw-rw-. 1 root kvm 10, 232 Jan 17 14:50 /dev/kvm

on a system that runs udev, you will probably need to add the following line in your udev configuration so it will automatically give the right group to the newly created device:

$ cat  /etc/udev/rules.d/80-kvm.rules 
KERNEL=="kvm", GROUP="kvm", MODE="0666"

8. To manage libvirt with a non-root account you should use PolicyKit. Define access control for a libvirt group:

$ sudo groupadd libvirt
$ sudo usermod -a -G libvirt bozz
$ logout

login again as bozz user and edit a new archive:

$ sudo nano /etc/polkit-1/localauthority/50-local.d/50-libvirt-remote-access.pkla

with this content:

[libvirt Management Access]
# For allowing access to specific user only:
#Identity=unix-user:bozz
# For allowing access to a group (like this guide):
Identity=unix-group:libvirt
Action=org.libvirt.unix.manage
ResultAny=yes
ResultInactive=yes
ResultActive=yes

restart libvirtd daemon:

$ sudo service libvirtd restart

verify if bozz user can locally access to qemu:///system (NOTE: the use of qemu:///session is discouraged):

$ virsh -c qemu:///system sysinfo
<sysinfo type="smbios">
...

verify if bozz user can remotely access to qemu+ssh://bozz@SERVER/system too. So on the Linux client issue:

for Ubuntu client (like in this guide):

$ sudo apt-get install libvirt-bin

for CentOS client:

$ sudo yum install libvirt

then:

$ virsh -c qemu+ssh://bozz@SERVER/system sysinfo
<sysinfo type="smbios">
...

change group ownership and permissions on the default images directory:

$ sudo chown root:libvirt /var/lib/libvirt/images
$ sudo chmod g+rw /var/lib/libvirt/images

9. [OPTIONAL] When libvirtd (>= 0.9.3) is running as non-root it tries to read ~/.libvirt/libvirtd.conf (see here). A workaround is: 

$ mkdir -p ~/.libvirt
$ touch ~/.libvirt/libvirtd.conf 

then issue a virsh command as bozz user:

$ virsh list
 Id Name                 State
----------------------------------

10. Configure Bridged Network by creating a new network script at /etc/sysconfig/network-scripts/ifcfg-br0:

$ sudo nano /etc/sysconfig/network-scripts/ifcfg-br0

and configuring the parameters according to your LAN settings (NOTE: options are case sensitive i.e. Bridge and bridge are two different options):

DEVICE="br0"
TYPE=Bridge
DELAY=0
ONBOOT="yes"
BOOTPROTO=static
IPADDR=192.168.11.12
NETMASK=255.255.255.0
NETWORK=192.168.11.0
GATEWAY=192.168.11.1
DNS1=192.168.11.2
PEERDNS="yes"
NM_CONTROLLED=no

then edit the Ethernet network script /etc/sysconfig/network-scripts/ifcfg-eth0:

$ sudo nano /etc/sysconfig/network-scripts/ifcfg-eth0

with the following content (NOTE: the hardware address depends on your NIC, an arbitrary MAC address is used here):

DEVICE="eth0"
HWADDR="00:2C:C2:85:29:A3"
ONBOOT="yes"
BRIDGE=br0
NM_CONTROLLED=no

  restart the networking service:

$ sudo service network restart

and verify the bridge config:

$ brctl show
bridge name bridge id  STP enabled interfaces
br0  8000.002cc28529a3 no  eth0
...

configure iptables to allow all traffic to be forwarded across the bridge:

$ sudo iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
$ sudo service iptables save
$ sudo service iptables restart

prevent bridged traffic from being processed by iptables rules, this improves the bridge’s performance. In /etc/sysctl.conf append the following lines:

net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

reload the kernel parameters configured with sysctl:

$ sudo sysctl -p /etc/sysctl.conf

restart the libvirt daemon.

$ sudo service libvirtd reload

Post-install Steps

• Creating a virtual machine / guest by means of Virtual Machine Manager (virt-manager) from the Linux client machine. The guest OS will be an Ubuntu Server 11.10 Oneiric Ocelot:


for Ubuntu client (like in this guide):

$ sudo apt-get install virt-manager

for CentOS client:

$ sudo yum install virt-manager

then download oneiric-server-amd64.iso from the Ubuntu site on the Linux client: 

$ wget http://releases.ubuntu.com/11.10/ubuntu-11.10-server-amd64.iso

copy the downloaded ISO to SERVER:

$ scp ubuntu-11.10-server-amd64.iso bozz@SERVER:/var/lib/libvirt/images/

on the Linux client run virt-manager:

$ virt-manager & 

goto "File" / "Add Connection..." with this settings: 

• Hypervisor: QEMU/KVM
• Method: ssh
• Username: bozz
• Host: SERVER

once connected click on the "Create a new virtual machine" button.

1. Set the name to ubuntu-oneiric and choose "Local install media (ISO image or CDROM)":

2. Browse and select the ISO located at /var/lib/libvirt/images/ubuntu-11.10-server-amd64.iso, OS type "Linux" and Version "Ubuntu 11.10 (Oneiric Ocelot)":

3. Choose memory and CPU setting as you wish:

4. Choose storage settings:

5. Choose the previously created bridge network device "Host device eth0 (Bridge 'br0')", Virt type "kvm" and Architecture "x86_64":

press "Finish" button and install the guest OS.



References

• KVM 
• KVM FAQ
• libvirt
• CentOS / Redhat: KVM Bridged Network Configuration
• HowTos/KVM
• Bridged networking with libvirt